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Foreword 


The field of International Development is a rapidly growing and challenging field. While the idea 
of poverty alleviation and economic growth has for remained a central concern for economists, 
concerns like income inequality, social inclusion, participation, transparency and accountability 
have become increasingly common ideas in many development for a including the World Bank. 
As these ideas are embraced and become integrated in practical development interventions, it 
is also necessary to review and understand how these ideas emerged or were first articulated. 
While some ideas came from academics and universities, many ideas and concepts became 
accepted as a result of the persistent struggles of practitioners in the field. COPASAH | 
(Community of Practitioners on Accountability and Social Action in Health) is a collective of 
practitioners who have been developing these ideas and applying these principles in the field of 
health governance in different places around the world. In these Issue Paper COPASAH members 
have deliberated over some of their key concerns to draw lessons for future practice. 


Health care is a contested area of governance and public policy action. It is also an area of 
immediate concern being featured prominently in the erstwhile MDGs and in the contemporary 
SDGs. In this series of Issue Papers, COPASAH members share their insights in critical issues 
especially related to the inclusion and participation of the poor and marginalised communities 
and how these may be negotiated or kept centre stage within contemporary development 
practice. The Issue Papers draw upon the years of practice of COPASAH members and are 
practical and insightful at the same time. We are sure these will provide important pointers 

for practice for any development practitioner in the field of heath governance. On behalf of 
COPASAH we look forward to your feedback and suggestions to continue the discussions and 
sharpen our practice. 
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TION | 


My personal acquaintance with community 


monitoring began around 1998 or 1999. | forget 
the year, but | remember the occasion well, even 
though it was a second hand experience. My 
colleague had just returned from attending a 
public hearing on panchayat-related! development 
activities in the Bhilangana block of Tehri Garhwal 
district? in the state of Uttar Pradesh (subsequently 
Uttarakhand) in India. She was full of stories 

of how during this event, people were giving 
testimonies of their names being misrepresented 
on labour muster rolls, of how people said that 
they had not received any money for any work, 
while the records showed that they had. This had 
led to the local gram pradhan(elected head of 

the village panchayat) and’thekedar’ (contractor) 
publicly apologising for these mistakes and 
promising to return the money. This work in Tehri 
Garhwal had been inspired by the process of Jan 
Sunwai (public hearing) that had been started a 
couple of years ago in Rajasthan by the Mazdoor 
Kisan Shakti Sangathan (MKSS). The process 

and the results that were obtained sounded very 
impressive. 


The mechanism of the Jan Sunwai later emerged 
as a very important tool for poor and marginalised 
communities to confront public authorities 

with questions about the performance of public 
systems. By 2005, the concept of ‘social audit’ 

had been legally included within the review and 
monitoring mechanisms of the Mahatma Gandhi 


National Rural Guarantee Scheme? (M NREGS) in 
India. 


During the same year, the Indian Parliament 
had passed the Right to Information Act (RTI), 
empowering the citizens to ask questions from 


aaaneninsentisneen 

1 ‘The panchayat is the lowest level of government in 
India. The gram panchayat or village panchayat is the 
elected committee at the village level entrusted with 
the implementation of various development activities 

2 India is administratively divided into States, Districts 
and Blocks. A block comprises of a cluster of villages 

. Any family which is registered under the scheme is 
entitled 100 days of paid work during a calendar year. 
The social audit process is meant to check whether 
the payments are made to the correct individuals 


and the work performed according to the norms and 
specifications, 


the public authorities. Both these pieces of very 
progressive legislations were to some extent 
influenced by the success and the potentials of the 


Jan Sunwai mechanism. 


My first experience of a ‘public hearing’ had : 
somewhat different results. It was in 2001 when | 
had resettled in Lucknow, the state capital of Uttar 
Pradesh. | was part of a group of health activists 
who were very upset that the government of Uttar 
Pradesh was continuing to pursue an aggressive 
and coercive family planning programme despite 
the fact that after ICPD (International Conference 
on Population and Development, Cairo 1994), the 
national government had committed to a Target 
Free Approach (TFA — 1996) and to voluntary and 
informed choice through the National Population 
Policy (NPP 2000)‘. In order to build a case against 
the existing state government's family planning 
policy framework, members of the network 
Healthwatch - UP Bihar, identified many women 
who were coerced through various means to get 
them sterilised. We were able to identify many 
women in whom the operation had failed and 

had become pregnant subsequently. We also 
conducted an opinion poll of elected panchayat 
representatives and other local leaders across ten 
districts to understand their opinion on how they 
felt family planning services should be delivered. 
One day in April 2001, a couple of hundred health 
activists and members of the community who had 
suffered from the coercive population policy of 
the state came together at the state capital. We 
wanted to share our story with the government 
authorities and ask questions. We presented 
secondary data, we presented the results of the 
opinion poll, and many of the victim/ survivors 
gave their own testimonies. Unfortunately, no 


representative from the government came to 


attend the hearing. The media reported the event 
widely, but as organisers we had a mixed sense 


4 UP is the most populous state in India with a present 
population of nearly 200 million. Population control 
in the name of family planning has for long been an 
obsession with successive governments in UP and this 
was also supported by international aid agencies. The 
USAID supported IFPS project has been underway in 


the state since 1992 and continued through till March 
2012. 


of success. We were not able to provide any direct 
benefit to any of the women or men who had come 
and shared their suffering. While there was a sense 
of catharsis, and of solidarity, we were not sure we 
had been able to engage the state. 


These anecdotes are intended as a preamble to 
the exploration of how we as practitioners can 
assess progress, outcome and impact of our work 
related to community monitoring. In the story from 
Tehri Garhwal there were immediate benefits to 
the people who had asked the questions, however 
in our own experience from Lucknow, we had no 
such luck. Even the World Bank now considers the 
instrument of social audit that was pioneered by 
MKSS in Rajasthan. Today, it is considered a potent 
method for ‘social accountability’ 


Many ‘jansunwais’ have been conducted across 
India, and some activists have even lost their lives 
in the process, but the situation of the poor has 
remained substantially the same in many of these 
locations. In our case, the experience gave the 
activists of Healthwatch UP and Bihar the strength 
to continue their advocacy on health rights further. 
It contributed to the development of a women’s 
mass organisation,Mahila Swasthya Adhikar Manch, 
in the state. It also contributed to the development 
of the community based monitoring methodology 
that was tested as a pilot intervention across nine 
states of the country through the National Rural 
Health Mission. 


The challenge for the practitioner is how to assess 
if the community monitoring approach in health is 
working and to find ways of distilling lessons, and 
to adapt and apply them in different contexts. The 
stories illustrate that the level and scale of influence 
of these processes can be vastly different depending 
upon timescale of review and the perspective that 
one adopts. How do we then judge progress and 
results? This paper will try to explore some of the 
mechanisms that can be adopted to ‘monitor’ 
progress, ‘evaluate’ success, and distil lessons. The 
paper is the result of the author’s own exploration 
of applying many of these concepts and thus 
represents a work in progress. 


Where do you want to go with 
Community Monitoring? 


My encounter with community monitoring is 

now about fifteen years old. | am still discovering 
new facets, challenges and possibilities. Broadly, 
‘community monitoring’ refers to initiatives 

where community members become involved in 
monitoring aspects of public service delivery. This 
could be through participation of parents in school 
management and oversight committees or patients or 
members of the community in hospital management 
and oversight committees. At the very simple level, 
the assumption is that the involvement of the 
community of users will make the services geared 
towards the need of the community and so the 
usefulness or quality and utilisation of the services 


will improve. However, this simple assumption often 


_ does not work. There could be many reasons for this. 


At a personal level, | have seen doctors who do not 
attend their official clinic, but run a private clinic close 
by. Recently, | was in a village and in my conversation 
with two young girls learnt that their teachers 

were not teaching them despite being present in 

the school. Many parents in that village preferred 

to send their children to a private school that runs 
opposite the road from where the government 
school is located. The World Bank devoted the World 
Development Report 2004 to investigate this issue. 


The report started with the assertion - “Too often, 
services fail poor people.” and in the same paragraph 
suggested ways of improving the situation — by 
enabling them (poor people) to monitor and 
discipline service providers. My own experience 

of trying to make very large and monolithic public 
systems like the health system work is not easy. 

On the one hand, these are huge and labyrinthine 
bureaucratic structures enjoying political patronage, 
while on the other hand communities often prefer 
a path of least resistance and seek the essential 
services elsewhere. Thus, to bring the two 

together ina relationship where the traditionally 
powerless community becomes enabled enough to 
‘monitor and discipline’ the public authorities and 
service providers is a most challenging but not an 


impossible t sk. 


Understanding the role of 
‘Community Participation 


The term ‘community participation’ seems a 
reasonably straightforward concept, but on 

closer examination appears more complicated. 

A recent World Bank report (Mansuri and Rao, 
2013) reviewing community participation in various 
development interventions found that ‘elite 
capture’ was a common phenomenon. However, 
this was something that we had learnt years 

ago through our own close association with the 
community. We had learnt that communities are 
not homogenous, even if they all happen to live in 
remote areas and share some common hardships. 
Some people have more resources and more local 
power and participation in decision making bodies, 
and these ‘elites’ are better placed to make use of 
the development benefits. Thus, if one is interested 
in the ‘poorest and most marginalised, one needs 
to be cautious of ‘elite capture’. We had also learnt 
that those who face the greatest deprivation do 
not necessarily have the greatest desire for change, 
thus one cannot assume that their interest or 
‘participation’ in efforts could possibly ‘change’ 
their reality, 


The word ‘participation’ also has many meanings. 
In one of the early discussion papers on this issue, 
authors Gaventa and Valderramma (1999) noted 
four linked approaches to participation. The four 
linked domains that they referred to were Social 
Participation, Citizen Participation, Political 
Participation and Participatory Methods. Much 
earlier, a Ladder of Citizen Participation (Arnstein 
1969) described - moving from manipulation 

to tokenism to citizen control. The possibility 

of change in the conceptualisation or purpose 

of participation has also been described as a 
move from ‘users and choosers’ to ‘makers and 
shapers’ (Cornwall and Gaventa, 2000). Further 
elaborating this shift from passive Participation 
to active engagement, Gaventa and Valderrama 
(1999) note domains of shift viz. from beneficiary 
to citizen; from a project perspective to a policy 
consciousness; from being consulted to becoming 


part of the decision-making process; from being 
included in the appraisal process to becoming 
organic to the implementation; and moving from 
micro to the macro. In short, shifts in these various 
domains taken together changes the context, the 
actions and the implications radically. 


Faced with this diversity of conceptual positions 
on just the concept of ‘community participation, it 
is difficult not to get lost. As a practitioner, when 
faced with this kind of conceptual labyrinth, | 

have tried to reconcile the ‘theoretical’ postulates 
with my lived ‘experiences’ and ‘instinctive’ 
feelings. What is it that has given me the most _ 
satisfaction in working together with people in the 
community? Have | seen any changes in the way 
they engage with ‘development’ processes? Has my 
understanding of ‘participation’ changed over the 
years? 


It doesn’t require long for me to understand that 

it has. | started my life as a rural doctor trying to 
‘do good’; educating people about healthy habits, 
providing curative health care to the poor at a very 
low cost. Soon | realised how little | knew about 
their lives and their realities and how my medical 
education didn’t equip me to communicate 

with my ‘patients’ with empathy or provide care 
which would sit easily within their existing life 
circumstances. Using participatory methods, | 

had started a joint enquiry with the women in the 
community on how women lived their reproductive 
lives. | learnt how they negotiated menstruation, 
childbirth and the post-partum period. Together, 
we explored stigma, safety, and bodily autonomy. 
Women in the remote rural mountains started 
scripting a new life for themselves challenging 
patriarchal norms of segregation and stigma while 
adopting safe health care practices. The four 


- linked concepts of participation that Gaventa 


and Valderrama have written about immediately 
start making sense. The movement of women’s 
engagement in their health care efforts from a 
passive participant (patient) to an active agent 
(similar to the progress on Arnstein’s ladder) is 
evident. 
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From ‘Participation’ to 
‘Monitoring’ 


According to different authorities, community 
monitoring or oversight of public services by 
citizens are possible when certain conditions are 
available within the governance structure of the 
particular country or state (NIAR (nd), Newell 
and Wheeler (2006), Potts (nd) quoted by Flores 
[AMHI/OSF 2011]). Some of these contextual 
conditions are as follows: 


VOICE - A strong, organised, citizen group that 
is able to articulate its opinion and position is 
called the citizen’s voice. It assumes mobilisation, 
knowledge about rights and entitlements and a 
confidence among the citizens to confront public 
authorities with their claims and experiences of 
deficits in public services. 


ACCOUNTABILITY FRAMEWORK - In order 
to ask questionsto the government or state it is 
necessary to have a framework on which such 
questions can be based. These can also be called 
the ‘rights’. While the constitution of a country 
often provides a framework, it is generally very 
broad. Frameworks that are more specific can 
be available within the programme and policy 
guidelines. In the field of health, even quality 

of care parameters and standard operating 


procedures adopted by the government authorities 


can serve as the accountability framework. 
Sometimes the accountability framework is drawn 
from international commitments that the state 
may have made for example, within UN treaty 
body frameworks. Implied within the concept 

of accountability framework is the concept of 
‘compact’ or the states assurance to provide the 


services. 


RESOURCES AND REMEDIES - Having a strong 
citizen voice and the rights or the compact may be 
good enough to start the process of community 
monitoring within a particular context, but it 

may not be sufficient to make it effective or to 
sustain it. Community monitoring will sustain if 


the community demands and claims lead to some 
form of change. In rights language, it is necessary 
for some form of remedies - redress or restitution 
to be possible through this mechanism if it has 


to sustain itself. This requires both political and 
financial resources. 


Purpose of Community 
Monitoring 


=» Community monitoring is a desirable act 
from the point of view of those who hold the 
community interests dear. However, providers 
and bureaucrats do not necessarily see it in the 
same way. In order to bring both parties to the 
same table, it is necessary to use frameworks 
of common interest. Some of the issues of 
common interest are: 


m Reducing corruption 
a Increasing quality and effectiveness of services 


= Improving effectiveness of development 
related investment or aid. 


Others argue that these are important but not 
necessarily the only reasons for enabling the poor 
and marginalised communities to assume charge 
of the process of monitoring. They argue that it is 
essential that the poor and marginalised become 
empowered to participate in decisions relating to 
their lives and participate in the political processes 
that govern their lives. This means community 
monitoring is part of a process to 


= Empower the poor. 
= Improve democratisation of public systems. 


= Increase peoples participation in policy making. 


Thus, community monitoring is an evolving 

field and the terminology is still fluid. It is given 
different names by different people. Many prefer 
to call it ‘social accountability’ while others call it 
‘demand for good governance. Still some see it as 
‘participatory governance’ and some would like to 
keep it bland and call it ‘community action. Often 
these different names indicate where we come 


from and where we would like to go, and when 


we have so many different names, it is possible 
that there may be some differences in our origins, 


approaches and destinations. 


However, as a practitioner one needs clarity 

of purpose. For me, the clarity of purpose 

is provided by my faith in the poor and 
marginalisedcommunities’ ability to aspire for, 
and their ability to create better conditions for 
themselves. | get inspiration from my belief in 
equity, social justice and human rights and in 
the shared access and control over the resources | 
available to all of us through principles of 
participatory decision-making. Thus, for me 
‘community monitoring’ allows for and depends 
upon changes in many local power relationships 
and my desired outcome is a situation where: 


= The poorest and most marginalised, especially 
women, are organised and empowered and 


they are aware about their health related rights | 


and entitlements. 


- They are able to use the public services 
without facing any difficulties. 
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HOW DO WE KNOW 
WE ARE MOVING 
AHEAD? WHAT TO 
MONITOR, HOW TO 
ASSESS PROGRESS? 
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Over the years, the nature of development, policy 
and interventions, have been undergoing many 
changes. On the one hand, the overall ideology 
of development has become a set of maneuvers 
between the practical paradigm of ‘Millennium 
Development Goals’ and the more aspirational 
‘human rights based approaches. On the other 
hand, ‘results’ and ‘evidence based’ policy making 
have become the rallying cry of funders as 

they try to ensure greater effectiveness of their 
development dollars. Thankfully, the twin issues of 
‘participation’ and ‘accountability are included as 
contemporary concerns, which allow community 
monitoring to have a place in the sun so to speak, 
but the current emphasis on results and evidence 
calls for the use of appropriate methodologies to 
demonstrate progress, results and effectiveness. 


Community Monitoring: A 
Complex Social Intervention 


As a practitioner, one needs to understand what 
the current situation is and then plan one’s 
intervention. Community monitoring can be 
considered a complex social intervention, where 
we are not only expecting a set of complementary 
outcomes but it also assumes that the 
interventions introduced through the project will 
set in motion some social processes, which will act 
synergistically to achieve the desirable outcomes. 
If women’s empowerment is a desirable outcome 
of a ‘community monitoring’ intervention, it is 
possible for the empowerment to manifest itself in 
different ways in different contexts. 


Some of these empowerment outcomes may be 
difficult to anticipate at the outset. This makes 
identifying all dimensions and measures of change 
difficult at the outset. Thus, understanding and 
assessing progress for community monitoring 
requires us to look beyond the commonly used 
methods for monitoring and evaluation. 


The usual way of monitoring progress of a 
development intervention is to develop a ‘log 
frame’ or prepare a chart of inputs and anticipated 


outcomes and set a list of indicators. This is 

the usual route-map and milestones method 
adopted by smart development practitioners. 
This approach is based on social and economic 
‘logic’, which has probably been tried and tested 
elsewhere and includes a set of contextual : 
assumptions. It is supposed to provide a simple 
and concise summary of what the project aims to 


do. It is systematic and logical and provides a basis 


for monitoring progress and results. However, 
the journey that we wish to undertake as 
practitioners of community monitoring with poor 
rural women is somewhat uncertain. At the same 
time we would like to see whether participatory 


- approaches are appropriate for developing 


citizenship aspirations within women; we would 
like to know if women’s action can influence 

the policy domain. At a more practical level, we 
also wish to demonstrate that these actions by 
women can improve their ability to receive good 


health care and improve their health status overall. 


In order that these outcomes are possible, it is 
necessary to build confidence among women 

so that they can engage with their health care 
providers. However, to do so they must be able to 
challenge many social boundaries at home and in 
their communities. 


The first question is have we ‘tried and tested’ 
the set of interventions that can ‘predict’ these 
outcomes? 


As | am aware, for ‘community monitoring’ 
there are no fixed formulae yet and there are 
contradictory results from randomised control 
trials’. Then what can one do? We can search 
online, talk to fellow practitioners, visit some 
field projects and in the end, we can have the 
beginnings of some answers. We can start our 
intervention and be vigilant about progress but 
the usual methods of project monitoring may 
not be adequate tools for charting our journey. 
We will have a level of uncertainty mixed with 
hope when we start. We are aware that there 


- Bjorkman and Svensson (2009) have shown 
community monitoring works in Uganda while 
Bannerjee and colleagues (2010) have not found 
similar results in India. 
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is a range of possibilities that could lead to the 
destination. We also have certain assumptions 
about social processes. Within this complexity, 
we also need to identify whether the movement 
of the intervention is in the desired direction. We 
need to learn about ‘progresses’ and pitfalls on 

a continuous basis so that appropriate changes 
can be made. What then can be the appropriate 
monitoring and evaluation methods that can 
support community monitoring? 


Alternative Approaches to 
Evaluation and Monitoring 


Huey Chen (2004), a leading evaluation expert 
has compared evaluation to fishing. Not only does 
one need a set of equipment, and the skill to use 
them, but one needs to know which method of 
fishing is appropriate under what circumstances. 
Wrong hook and wrong bait in the wrong 

place could be the difference between success 
and failure. Community monitoring has been 
mentioned as a ‘complex social intervention. The 
Medical ResearchCouncil of UK® defines these ‘as 
interventions with several interacting components’ 
and acknowledges that ‘they present a number of 
special problems for evaluators. 


Many of the extra problems relate to the difficulty 
of standardising the design and delivery of the 
interventions, their sensitivity to features of the 
local context, and complexity of the causal chains 
linking intervention with outcome. Following 

this advice, the common practice of comparing 
post-intervention status to a pre intervention 
baseline to find the change, commonly called 

the summative form of evaluation or outcome 
focussed evaluation, is not of much use for our 
purposes. 


For our purpose, an evaluation method that 
allows the process to be examined is more 


6 Developing and Evaluating Complex Interventions: 
-New Guidance, Medical Research Council, 2008 
~ available at http://www.mrc.ac.uk/Utilities/ 
Documentrecord/index.htm?d=MRC004871, 
accessed 19th October, 2010 


appropriate and this is where the concept of 
‘programme theory’ is useful. Carol Weiss has 
argued in ‘Nothing as Practical as Good Theory’ 
(Weiss 1995) that ‘social programs are based on 
explicit or implicit theories about how and why 
the program will work’. Taking this idea forward 
Pawson and Tilley (2004) say that all social 
programmes are shaped by the ‘vision of change’ 
and are ‘hypothesis of social betterment’. Realist 
evaluation tries to understand the core theories 
behind an intervention and then interrogate 
whether these theories work, under what 
circumstances and for whom. In their conceptual 
framework, the context and mechanisms of 

the interventions interact differently under 
different circumstances and the outcomes 

can be different for the same mechanism of 
interaction under different contexts. Realist 
evaluation ‘seeks to unpack the mechanism of 
how complex programmes work (or why they fail) 
in particular contexts and settings’ (Pawson et al 
2005). The realist approach tries to understand 
the mechanism (M) which relates the outcome 
that is desired (O) to the context (C) in which 
the intervention is applied. It allows for an 
interrogation of the role of the context, is open 
to a range of outcomes and tries to understand 
the mechanism that is articulated as a series of 
hypothesis or theories. Clearly, this method of 
evaluation which allows us to have an imaginative 
vision and which is rooted in the context is much 
more appropriate for our monitoring-adapting- 
learning—evaluation process. 


Another approach to evaluation, which appears 
appropriate for community monitoring, is 
Development Evaluation. It is pioneered by Patton 
(2008) and is an approach that allows learning and 
innovation to take place concurrently with the 
implementation process. This concurrent learning 
process is crucially important to the community- 
embedded practitioners who cannot fail and 

walk away, or even succeed and then ‘close’ their 
practice. Learning lessons, from success and 
probably more from failures is necessary for the 
practitioner to continually improve their praxis. 

In many situations, the practitioner cannot watt 


a 


until the “end” of the programme, but needs 


information on a real-time basis to solve emerging 


problems within the overall programme. 


Developmental evaluation differs from traditional 


forms of evaluation in several key ways: 

= The primary focus is on adaptive learning 
rather than accountability to an external 
authority. 

= The purpose is to provide real-time feedback 
and generate learnings to inform development. 

= The evaluator is embedded in the initiative as a 
member of the team. 

= The evaluator role extends well beyond 
data collection and analysis; the evaluator 
actively intervenes to shape the course of 
development, helping to inform decision- 
making and facilitate learning. 


dynamics and surface. 
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MOVING FROM 
THEORY TO ae 
PRACTICE | 


| hope | have made a case that it is not only 


possible but also necessary to use some method 


for assessing change and understanding progress 
in our work around community monitoring. We 
can do so by using approaches that allow us to 
learn, adapt and improve while at the same time 
keep us grounded to our overall purpose and 
principles. However, before we start the process 
of monitoring or evaluation or assessing progress, 
we have to be extremely clear where we stand and 
where we want to go. In other words, we need to 
clarify the conceptual frame of our intervention 
and develop our programme theory. This process 
forces us to think through the purpose, and goal of 
our intervention in the broadest possible manner 
and allows us to be expansive in our goal setting. 
However, in order to stay rooted we need to also 
consider our own unique contexts while we are 


engaged in this exercise. 


Mention has been made at the outset that different 
people can have different expectations from 
community monitoring. We need to have clarity 
about the balance between improving effectiveness 
aspect and empowering citizens/ deepening 
democracy aspects that we seek through our work. 
We also need to have some idea about the levels 

of effect or impact we seek through our work. Is 

the result anticipated at the level of the community 
and their health, or in the levels of organisation and 
empowerment? Is the result expected at the level of 
health status, delivery of services or the attitude and 
behaviour of providers? 


Similarly, what is the balance of our interest in 
specific changes in one area and changes at the 
level of development discourse and practice? To 
give an example, a large-scale development project 
promoted by the government or by an international 
donor may be interested in improving effectiveness 
reducing corruption and in specific results on the 


ground. However, a non- governmental intervention 


) 


started over a small area may be more interested 

in strengthening voice of the local community, 
demonstrate some change and then advocate with 
the state government to introduce this component 
as a regular monitoring mechanism in all public 


service schemes. What is it that we want? It is useful 
to have some clarity when we start our work. 


Developing an Appropriate 
Programme Theory 


Having developed clarity of purpose of our 
intervention, we need to develop the ‘programme 
theory’ of how the desired change will be achieved. 
For example, if our desired status is one of an 
empowered and mobilised community, we need 

to understand their current levels of mobilisation 
and propose a mechanism for improving the same. 
Similarly, if we want that communities are able to 
speak freely with their health providers about the 


_ problems they face, we will need to explore the 


current status of their relationship and propose a 
method through which it may be improved. As we 
seek answers to these questions, we will also need 
to find methods that we can adopt in our practice 
to complete our programme theory. 


One of the first considerations in developing 

the appropriate programme theory is to explore 

the context in which the intervention is applied. 
According to discussions at a recently held meeting 
of practitioners of community monitoring, which 
has been summarised by Kaim (AMHI/OSF 

2013), the context can include both the larger 
environmental context as well as the institutional 
context of the organisation that is engaging in 
promoting the exercise. The environmental context 
is further divided into the political context, which 

is important for determining the extent to which 
this kind of citizenship promoting exercise will be 
‘allowed’, the health system context with which | 
the engagement is expected to take place and the 
cultural context of the community that is expected 
to play a key role in this monitoring exercise. The 
institutional context includes the institutions vision, 
its relations with the community as well as its 
strategic advocacy focus and last but not the least, 
the skill sets it possesses. The institutional context 
clearly determines the approach that it will adopt to 
promote the engagement of the community with 
public authorities. 
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Sridharan and Nakaima (2011) have provided some 
practical steps for applying the realist evaluation 
methods and | have found their approach very 
useful7. The first four steps of their proposed 
ten-step approach have been very important for 


us in developing programme theory related to our 
interventions. They ask: 


a What are the different components of the 


programme? Are these components stable? 
a What is the programme theory? 


= Does the evidence base support the 
programme theory linkages? 


a What is the anticipated performance 
trajectory? 


While the journey of community monitoring 

can be aspirational and the path still not well 
known, these four questions allow us to set up 
our compass and give us an idea of what we are 
looking for so that we know we are going in the 
right direction. In order to develop an appropriate 


Key Considerations 


Voice — Current status of 
mobilisation among the 


Assessment of Current Status 


We will need to understand - 


What is the current state of mobilisation of the 


intervention for community monitoring based 
on sound programme theory, | have found the 


following analytic approach useful: 


Sridharan and Nakaima suggest that the list of 
programme components that we finally select as 
part of our intervention package must have some 
evidence that they have worked somewhere so 
that we have some idea about its stability. We 
can develop our programme theory by linking the 
three columns of the table to create an appropriate 
hypothesis about the change we expect that 

the programme component will bring about. 

The linkages we choose will depend upon our 
context as well as our interest. As we develop the 
linkages and finalise our programme components 
and a sequence of action, we develop a possible 
performance trajectory. The methods that we 
identify also provide clues about the interim 
processes through which we anticipate that these 
changes will take place. 


Possible Programme 
Components 


Mobilisation Awareness 
raising, Conscientisation. 
Community action 


Policy Analysis 


marginalised community? 
What are the levels of hierarchies within the 
community? 
Who are the most marginalised? 
Are they aware of their entitlements? 
What are the levels of leadership and 
autonomy? 
Are there groups and cliques? 
Accountability Framework What are the current levels of service 


delivery? What are the key gaps? Is there 
data available about these gaps? Do the 
community or primary users see these 
as gaps? What are the data disclosure 


Are there laws, policies and 
programmes and budgets 
that provide necessary 


Secondary data analysis 
Research 


Social Audit 


services as entitlements. 


Has the country made 
any international 
committments. Are 
there standard operating 
procedures, quality 
standards for service 
delivery? 


procedures? Are there laws around 
information disclosure? 


Are the service providers aware of the state 
laws and policies and their responsibilities? 
Are they provided adequate resources? 


What are the current monitoring mechanisms? 
Are these provisions for community to 
participate in these mechanisms? 


Participatory Research 
(PRA/ PAR) 
Enquiry/Investigation/Fact 
Finding 

Score cards/ Report cards 
Expenditure tracking 


Complaints/ Grievance 
collect 
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Key Considerations 


Resources and remedies 
for change Is there 
interest in change? Are 
there adequate human, 
financial and material 
resources for change? 
Are there provisions and 
mechanisms for changes 
and remedial action? 


Setting the programme theory and anticipating the 
trajectory of change is essential to the monitoring 
and evaluation process. The practitioner now 

has to be vigilant to understand the processes as 
they unfold and to take stock from time to time. 
As a practitioner, | have found the concurrent 
documentation and analysis of stories of ‘change 
and resistance’ from the field to be very valuable 

to track progress and understand change and 
assess trajectory. Other processes of monitoring 
and review like stakeholder meetings, third party 
assessments can all be useful for understanding 
whether the programme theory holds and whether 
the components are working as assumed. They 
help us to make necessary changes and to draw 
lessons about possibilities and limitations. 


Both qualitative and quantitative methods can 

be used to understand how close far we are to 

our expected goal, but we need to be constantly 
aware of unanticipated processes that can be both 
beneficial and problematic. 


The approach that Sridharan and Nakaima 
Propose not only allows us very practical tips 

for starting our work, but also provides ways 

in which we can try to cull lessons to improve 

our programme. They acknowledge that most 
evaluators are concerned with ‘effectiveness’ while 
the practitioner may require methods that ‘follow 
a continuous improvement model’. They contend 
most evaluation assumes that interventions and 
contexts as static and stable whereas in real life 
this is often not the case and it is necessary to be 
adaptive. They also raise questions about the role 
of evaluation in the context of spread, scalability 
and generalisability. 


Assessment of Current Status 


What are the areas of concern for local 
authorities and service providers? Can they 
be allies? What are the larger political/ 
programmatic concerns and compulsions? 


Possible Programme 4 
Components 


Advocacy with authorities 7 
Public Dialogue 

Public Hearing 

Committee Hearing 

Legal Action 


Drawing Lessons from Existing 
Practice 


In this section, | will try to apply some of these 
premises to existing practice of community 
monitoring. | will limit this application to two 
examples from India, with which | am more 
familiar. Case studies based on these two practices 
are prepared separately by COPASAH and are 
readily available to the interested reader. One 

of the cases is that of the Community based 
monitoring (CBM) of National Rural Health Mission 
in the state-of Maharashtra and the other is that 
of the Mahila Swasthya Adhikar Manch (MSAM 

or Women’s Health Rights Forum) in the state 

of Uttar Pradesh. While | am offering these as 
examples, | am aware of major limitations of this 
exercise. While | have discussed the use of these 
cases in this paper with the key functionaries of 
the two organisations, my dataset of information is 
extremely limited. 


Context 


Common Environmental Context -The 
Government of India launched the National Rural 
Health Mission (NRHM) in 2005 as a new and 
integrated approach for providing health care 
services to the rural poor in India. The design 

of NRHM was made with the participation of a 
range of civil society public health experts and 
health activists within the overall mandate of 

the Common Minimum Programme, a common 
agenda of governance adopted by the UPA (United 
Progressive Alliance) Government. Community 
Participation and strengthening of monitoring and 


accountability mechanisms were included as key 
strategies within this overall approach. An advisory 
committee on community participation called the 
Advisory Group on Community Action (AGCA) 
was also included within the overall planning and 
review processes of the NRHM. This committee 
comprises solely of civil society members, and 

it was instrumental in designing and rolling out 
the pilot phase of Community Monitoring. This 
pilot phase was rolled out across nine states 
between 2007 and 2009. Subsequently, some 
state governments continued with community 
monitoring while others initiated the process. 


The process of community monitoring depends 
upon the constitution of a set of planning and 
monitoring committees from the village upwards 
to the state. They start at the level of the village 

as the Village Health, Sanitation and Nutrition 
Committees and these committees are expected 
to conduct an enquiry into the status of health 
services received by the community, especially 
women and children and draw up a report card. 
Similarly, they are also expected to review the 
services available at the health centres and draw up 
another facility report card. These two report cards 
form the basis for a periodic dialogue with health 
care functionaries and authorities at different levels 
to identify improvements and remedial action. 


CBM-Maharashtra: The community based 
monitoring process in Maharashtra was part 

of the larger nine state pilot project that was 
implemented nationally with the support of the 
Ministry of Health, but this exercise in Maharashtra 
had some unique aspects.. The implementation 
of CBM in Maharasthra was supported by the 
voluntary organisation SATHI, which was involved 
in developing the methodology at the national 
level as member of the AGCA. It was also a key 
constituent of Jan Swasthya Abhiyan (}SA), which 
had organised the Right to Health Campaign in 
2003-04. SATHI had prior experience of using the 
method of public hearing or Jan Sunwai through 

a series of such events that had been organised 

as part of the Right to Health Campaign. Some 
of these public hearings were conducted in 


collaboration with the National Human Rights 
Commission (NHRC), and this allowed people 
who had been denied health services to make a 
direct call to state authorities for better health 
care services. SATHI is implementing the initiative 
with a large number of partners who facilitate the 


different processes at the district and sub-district 
levels. 


MSAM - Uttar Pradesh: Mahila Swasthya Adhikar 
Manch (MSAM) is a statewide women’ s group 
with over 11,000 members, structured into village 
level groups, blocks, and district level federations 
in the state of Uttar Pradesh (UP). It is facilitated 
by some voluntary organisations and coordinated 
by SAHAYOG. SAHAYOG has been working 

on women’s health rights and accountability 

since 2001. SAHAYOG and its partners are part 

of Healthwatch Forum UP, a health advocacy 
platform that had engaged in many health 

rights campaigns, including the Right to Health 
Campaign with JSA. MSAM was initiated in 2006 
as a follow up to a state wide campaign called Puri 
Nagrik, Pura Hag (Complete Citizens Full Rights) 
which had engaged thousands of women across 
the state. After NRHM was announced, SAHAYOG 
and its partners found that there were spaces for 
communities to engage with the health system. 
However, Uttar Pradesh was not included among 
the states included in the pilot CBM under NRHM 
and MSAM led advocacy for better health services 
has not happened within any prescribed NRHM 
platform for engagement. 


Processes and Mechanisms 


CBM-Maharashtra: The CBM process followed 

in the state of Maharashtra follows the basic 
approach prescribed under the national guidelines. 
The process includes preparatory activities like 
creating an environment and getting a mandate 
from all stakeholders, formation and capacity 
building of the different committees, a process of 
community enquiry to obtain community feedback 
of services that are summarised through a report 
card, and finally a public hearing or Jan Sunwai. 


The Jan Sunwai is used as the key tool for ensuring 
public accountability. The Jan Sunwai's allow 7 
people to interact with government health officials 
in the presence of local legislators, members of 
the panchayat, as well as an independent panel 

of judges. It has also emerged as a media event. 
The Jan Sunwai is seen as a key mechanism 

for deepening democracy in the CBMP in 
Maharashtra. It has been used since it ‘generates 
egalitarian aspiration among the marginalised’, 
enhances ‘confidence of the oppressed, and 
reduces hierarchies by challenging the ‘remoteness 
of the bureaucracy. Over 200 Jan Sunwais were 
held in various districts between 2008 and 2012. 


MSAM - Uttar Pradesh: The community 
monitoring process is not limited to health 

services and involves other social sector services 
for the poor including the pre-school services, the 
subsidised ration shop services, pension scheme for 
the old and the rural livelihood programme. The 
focus of the intervention is to build capacity among 
women leaders of MSAM groups to become aware 
of their own entitlements and the responsibility of 
the public system. They are trained to identify gaps 
in service delivery and to address public authorities 
with their grievances and issues. The women 
leaders have conducted community level enquiry 
process of a different scheme every year and the 
methodology used was based on participatory tools 
that can be used by the non-literate. The results 
are shared with district level authorities. There is 
also a regular series of village, block and district 
level meetings where women discuss issues related 
to their ongoing experience of service delivery. 
SAHAYOG has been organising annual state level 
dialogues as well as occasional interface and public 
hearings with senior public functionaries. 


Outcomes 


The following section very briefly tries to capture 
the changes at different levels, with some 
emphasis on the difference between the two 
efforts. Additional details about both these efforts 
are available as companion case studies. 


Changes in Community 
Empowerment 


CBM-Maharashtra: The key community level 
players were expected to be the members of the 
VHSNC members. Their involvement has been 
uneven across the different districts. In many cases, 
members of community groups organised by the 
NGO facilitators are more active in this process 
than the VHSNC members themselves. Leaders 

of the local government institutions have started 
taking an interest in this process and now they play 
a significant role in ensuring action by the health 
system in some areas. There has also been an . 
increase in the community’s interest and awareness 


about health, and an increased solidarity among 


the community. 


MSAM - Uttar Pradesh: The most important gain 
through this process has been the empowerment 
of the women leaders of MSAM who are almost 
exclusively from the most marginalised groups in 
the community. Many of the empowered women 
leaders have also started participating in the local 
government elections and processes. 


Changes in Community - 
Provider Relationships 


CBM-Maharashtra: The providers and the 
community are mandated to face each other 
through the process of public dialogue at different 
levels and so it has improved accountability of 

the community-provider relationship at different 
levels. Enhanced accountability has been linked 
with improvements in delivery, quality and 
outreach of services in several areas, which is 
reflected in numerous ‘stories of change”. However, 
the effect of the interactions reduces as we look 

at higher levels of governance. Positive changes in 
community-provider relations are seen mostly at 
village, PHC and block levels. State level interfaces 
have been more difficult to organise, although 
higher-level officials have been relatively supportive 
of accountability processes at lower levels, 


MSAM - Uttar Pradesh: The community- 
provider relationship is often adversarial at the local 
level. There is no formal forum for interaction and 
most interactions with providers take place by the 
mediation of the facilitating NGOs. The interaction 
of women leaders with public functionaries at the 
state level is more ‘ceremonial’ than substantive. 
However, the women leaders have gained greater 
credibility within their villages and with health 
providers. The facilitating organisations have also 
become more credible and are invited to formal 
platforms at the district and state levels. 


Changes in the Health Care 
Services 


CBM-Maharashtra: The Jan Sunwai process in 
Maharashtra has led to many improvements in 
service delivery. These improvements are mostly 
in response to gaps identified at the local level and 
have ranged from increase in outreach services, 

to completion of unfinished construction work, 

to better maintenance of hospitals, to starting of 
mandated surgery services. 


MSAM - Uttar Pradesh: There have been some 
changes in service availability at the peripheral 
level. Women leaders have often used their 
collective identity to demand for better services. 
However, there is no evidence pointing to systemic 
changes in the way services are delivered because 
of MSAM feedback. 


Other Gains 


CBM-Maharashtra: This process continues to be 
an example and inspiration for community based 
monitoring processes in the health sector within 
the country. This example was demonstrated to 
the Planning Commission of the country as an 
example of success and is proposed as standard 
practice within the 12th Five Year Plan framework. 
It is documented and discussed within the 
COPASAH for learning purposes. 


MSAM - Uttar Pradesh: This process is 
documented and disseminated through various 
means, providing inputs to the development of 
the policy framework for community involvement 
within the recently released report on Universal 
Access to Health Care, prepared by the High 

Level Expert Group constituted by the Planning 
Commission of India. Like the CBM - Maharashtra 
process, it is also being documented and discussed 
within the COPASAH for learning purposes. 


Challenges/ Disappointments 


CBM-Maharashtra: Though it is an officially 
mandated programme, the higher levels of 
bureaucracy do not pay adequate heed to the 
suggestions and recommendations that emanate 
from below. The process has not been able to 
address any of the structural gaps that affect 
service delivery, for example, staffing/ posting, 
equipment/ supplies etc., which require system 
wide changes. This has led the organisers to 
identify categories of ‘CBM sensitive’ as well as 
‘CBM resistant’ problems. Secondly, while the 
process has been able to identify area-specific 
problems and suggest solutions, these insights and 
recommendations have not found much reflection 
in the planning process, which is. supposed to be 
through a bottom-up district centred approach. 
Thus, the CBMP feedback has been able to 
influence the Rogi Kalyan Samiti (Patient Welfare 
Committees) in their facility level planning in some 
areas, while there has been hardly any dent in 
District PIP formulation. 


MSAM - Uttar Pradesh: The systemic changes 

in health service delivery have been far less than 
the community level impact of this endeavour. 

The community empowerment process has led to 
many women leaders joining the formal electoral 
process. Some of these leaders are continuing to 
work together with the women’s groups. However, 
this is not uniform and in some cases, these leaders 
have become co-opted by the existing vested 


interest groups. 


17 


Ongoing Documentation and 
Learning Mechanisms 


Both these experiences have had a very strong 
ongoing documentation process. In both cases, 
there has been an active process of collecting 
‘stories’ of change. In the case of the CBM- 
Maharashtra, these stories are regularly compiled 
and published in a quarterly state level newsletter 
‘Davandi’ In the case of MSAM- Uttar Pradesh, 
compilations of stories are published and are 
systematically used as learning material Our 

Story in Our Words (Humari Kahani Humari 
Zubaani). Both efforts have produced video films 
documenting their experiences and learning. 

There are platforms where the various partners 
come together periodically to review progress 

and incorporate changes. A number of formal 
evaluation studies are commissioned in both cases 
and lessons emerging from these incorporated into 
practice. In both cases, the senior most level of 
leadership within the coordinating organisation has 
been playing an active documentation and learning 
role within the programme leaving the task of 
coordination and implementation to others. This 


has allowed a robust and active learning process 

to emerge concurrently with the implementation 
process. These senior functionaries have also been 
very active in advocating for this approach within 
different policy spaces, multiplying the influence of 
the intervention manifold. 


Summary - The two interventions described above 
have used a similar bottom-up accountability 
approach to improve health services delivery. 

Even though some of the contextual factors are 
similar, there are differences in the institutional and 
environmental context in both cases. The overall 
programme theories too have both similarities and 
differences. The key intervention is different in the: 
two cases - in one case it is the jan sunwai and 


‘in the other it is capacity building among women 


leaders. Both efforts have been using a vigorous 
documentation process through which they are 
able to chart progress and incorporate ongoing 
learning within the programme implementation 
process. The results too have similarities and 
differences, which are not difficult to understand 
once one has gone through the entire sequence. 
One could even say that the expectations from 
these two processes should not be similar. 


BEST PRACTICE | 
AND SCALABLE 
INTERVENTION 


Today, we are faced with the paradigm of ‘evidence 
based’ practice and policy making. A development 
agency that has the mandate and purpose of 
stimulating and supporting socio-economic change 
over large geographies and multiple countries is 
anxious to learn of ‘best practices’ and ‘scalable 
solutions’. Even though we may not be part of 
these larger bodies, we may need to respond 

to their interests. On first sight, this approach 
seems to make sense for large-scale development 
practice. They want to repeat past successes and 
this approach appears to provide some assurances 
of success, especially if a rigorous experimental 
study is seen as successful. However, on closer 
examination, this assertion does not seem a 


rigorous one to me. 


| am not among the large-scale implementers of 
development projects. | started my life as a small- 
scale innovator and then became interested in 

the larger development interventions because 
from where | was located they did not appear to 
deliver what they promised. Furthermore, this 

gap between promise and actual practice had 
disastrous consequences for the lives of the poor. | 
have seen this time and time again in many health 
interventions — some of which have also received 
international plaudits including the Global Polio 
Eradication Initiative and the JananiSurakshaYojana 
(Mother's Safety Scheme - or the maternal health 
initiative of the Government of India). What | 
learnt from my close observation of these gaps 
was that the original project did not have a clear 
articulation of ‘why’ it should succeed in the 
different contexts that it was being implemented 
in. Thus, it could have benefited from a well 
articulated ‘programme theory’. Nevertheless, what 
was more serious was that there was no ongoing 
collection of information on what happened 


within the community once the programme was 
implemented. 


As we are moving into an increasing quantitative 
understanding of results and evidence, programme 
managers fret about numbers, often promoting 
misreporting, and no one even raises questions 


about the possibility of things going anywhere else 
but what has been ‘predicted’. Most large-scale 
projects seem to be based on the naive assumption 
that when interventions will be introduced 

people will start using them out of self-interest. 
Moreover, if self-interest needs to be generated, - 
some cash incentive will do the trick, or else 

some arm twisting may be necessary. Most of the 
information collected in the case of the large-scale 
projects | mentioned above was of the delivery 

of inputs with little or no attention on how the 
inputs affected different communities, including 
the relationship between different social groups 
and between providers and the community. What 
was perhaps worse was the lack of openness to 
hearing any other possibility than what had been 
written in the ‘books’. To me this appears not only 
as a lack of openness to learning but as a form of 
‘fundamentalism’. 


While community monitoring is not in the same 
category of interventions like the polio eradication 
or maternal health service delivery, the close 
observation of these large-scale programmes 

has provided some lessons for me. Community 
monitoring is perhaps an even more complex social 
intervention,as in this case there is no technical 
product that is expected to deliver results. Thus, it 
is extremely important to develop an appropriate 
‘programme theory’. To do so, one needs to define 
the context and | have already tried to show how 
one can develop it based on what Sridharan and 
Nakaima have proposed as stable ‘components’. 
Once the homework is done, we need a 
mechanism that will allow us to test the hypothesis 
in an ongoing manner. These interventions are 

not only complex because they have interacting 
components in their conceptual design, but also 
because they will ‘upset’ the power equilibrium 
existing in society, between members of the 
community and between the community and the 
health providers. We cannot exactly anticipate how 
an individual will react to a challenge to their social 
position. We need to observe, and we may need to 
adapt. This requires what may be called an active 
‘intelligence’ gathering and processing mechanism. 


This is where | feel stories and their ongoing 
analysis is important. 


Through my own work as a practitioner in areas 
of emerging development practice, | feel we do 
prepare a mental map, and collect and analyse 
stories. We do it ‘mentally’ and intuitively. We 
need to be much more deliberate and this will 
sharpen our own practice into realm of ‘evidence 
based. It will also allow us to be more deliberate 
in drawing lessons, relating to the context and 

to the specific components of our interventions. 
In this way, we can avoid the pitfall of proposing 
‘best practice’ type of universal solutions. 
Therefore, my recommendation is that we should 
be very deliberate in writing our assumptions and 
hypothesis about our interventions. We should 
write up what it is that we aim to change and 
how we think this is going to happen. We should 
be explicit about the different components of 
our programme - why we think they will work. 
We should be clear from where have we derived 
this anticipation of success, and this may be from 
our own earlier practice. If possible, we can draw 
a diagrammatic vision of the proposed societal 
interactions and routes of change. The more 
explicit we are in charting the possibilities of our 
intervention, the more accurate we can be later in 


understanding whether these were correct. These 


specific anticipations also work as our milestones 
during the process of review. 


The second area which | feel needs strengthening 
is our process of understanding what is happening 
in the community once the interventions have 
begun. We need to understand the dynamics of 
social interactions that have started as a result of 
our interventions. We not only need to understand 
what is happening as ‘events’ but in terms of 
‘sequence’ and ‘consequence’. | have found stories 
to be very useful for this purpose. However, we 
need to train our field workers to collect sufficient 
details and this can build their skills as community- 
based ethnographers. It is best if these stories 

are dated and analysed at different points of time 
and also in a sequential manner. They can provide 
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interesting insights about the time trajectory of 


social change that Sridharan and Nakaima have 
alluded to. 


Community monitoring is a dynamic learning 
process. As facilitators of this process, we are 

not the only party that is learning and changing. 
The community mobilisation and empowerment 
process is expected to increase the community's 
autonomy and make them the key players in 

the intervention, marginalising the role of the 
facilitator. Similarly, the public authorities are not 
passive entities either. They will continually try to 
identify their interests and act accordingly. Thus, 
in a process of documentation or evaluation of 
such a dynamic inter-dependent learning process, 
it is necessary that the key stakeholders also 
become part of the evaluation process, not only as 
respondents but as active participants. 


To summarise, it is helpful for any practitioner to 

prepare: 

= Awritten summary of their assessment ofthe 
environmental context, including a policy/ 
programme analysis identifying accountability 
frameworks and possibilities for changes. 


= Anassessment of their own institutional 
context and a plan to draw on existing 
resources where there are capacity gaps 


= Aprogramme theory that includes a 
series of hypothesis of why they think 
their programmatic approach or different 
programme components will work. 


= Adiagrammatic representation of what are 
the direct and anticipated interactions that 
will take place between different actors/ 
stakeholders and factors (community, cultural, 
programmatic, service conditions). This can be 
both beneficial interactions and resistances. 

= Alist of possibilities for intermediate situations 
through the programme theory articulation 
and diagrammatic representation. 


= Adocumentation and assessment plan for 
how and when the comune tae ae process will 
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to regularly review the plan to identify lessons 
and modify interventions. 
= Documentation of the review processes and the 


decisions. 


One can locate specific studies and even 
quantitative assessments within this framework. 
However, it is useful to remember that changes 

in power relationships can take considerable time 
to take place and initial changes can be difficult to 
sustain. Therefore, an interpretation of quantitative 
results needs to be nuanced with an understanding 
of the change processes within the community and 


between the different stakeholders. 


Project managers are often anxious in trying to see 
if the various project inputs are delivered in time 
and managing different practical discrepancies. In 
order to strengthen the learning component of our 
intervention, it is important to have an ‘embedded 
evaluator’ in the manner proposed by the Patton 
and colleagues of the Development Evaluation 
discipline. This person has to be sufficiently close 
to conceptualdimensions of the process to be 
keenly interested and reasonably distant so as not 
to be caught up in the day to day implementation 
and sufficiently respected for their analyses to be 
taken seriously by the day to day managers and 
implementers. We also need periodic opportunities 
where the documentation of progress may be 
reviewed and lessons drawn for incorporation 

into the ongoing implementation process. Our 
implementation process should be flexible 

enough not only to make changes to adjust, but 


to adapt, learn and grow. Many of us may not be 
in a position to hire a professional Development 
Evaluator, but we can certainly find colleagues 
who could provide this function. We may need to 
involve such colleagues right from the conceptual 


stage, or as early as possible. 


| have highlighted the importance of thinking- 


_reflecting- writing throughout this process - from 


the context to the planning process; from the 
regular ongoing documentation and stories of 


_ change and resistance to the report of review 


process; from the positive-outcomes to the 
lessons learnt and challenges faced. This body of _ 
documentation becomes the source of a case study 


_and the body of new evidence that is sought by 


the large-scale development project implementer 
and policy maker. It is comparable, if not superior 
to the experimental study even though it does 
not vouch for generalisation of the solution nor 
provide sophisticated statistical analyses. Instead, 
it provides an idea of not only what works and 
does not, substantiated by evidence and logic. 

It also provides insight about how the processes 
take place within a community and how the 
context may interact in particular ways with our 
interventions. It could also provide us more than 
one way to approach the situation and different 
ways of interpreting success and possibilities. It 

is a tragedy that we seek uniform solutions in the 
vastly diverse world of ours, and this approach to 
monitoring and evaluation can promote this spirit 
of diversity. 
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